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Abstract 

We propose a model of emergence of cooperation in evolutionary games that high- 
lights the role of network formation and effect of network structure. In line with 
empirical data, the model proposes a mechanism that explains the persistence of 
heterogeneous types (heterogeneity in rules for changing behavior) within a popu- 
lation, and in particular the sustainability of altruism (presence of unconditional 
cooperators) even in case of strong social dilemma. This explanation constitutes 
an alternative to the choice and refusal mechanism that is often presented as an 
explanation for cooperation on dynamic networks. We also exhibit a minimal set of 
strategies for emergence of cooperation (altruists, reciprocators and selfish agents) 
and sketch a two steps scenario for this emergence. We adopt an hybrid methodology 
with both analytical and computational insights. 

Key words: prisoner's dilemma, cultural evolution, endogenous social networks, 
selection of interactions, evolutionary games, heterogeneous agents, heterogeneous 
equilibrium 



Introduction 

Emergence of cooperation under natural selection pressure as long been a puz- 
zle for evolutionary game theory, which traditionally considers agents interacting 
at random within the population. However, new insights have been obtained since 
modelers recently started to take into account networks and their dynamics in the 
modeling of interactions. Besides structural and topological constraints that deter- 
mines the possibilities and conditions of interactions (who can interact with who), 
activity networks (who is actually interacting with who) are emergent patterns of 
social activities. People meet each other, interact, learn from these interactions and 
eventually keep previous partners in mind as potential future partners. These emer- 
gent networks from agent's activities strongly influence their outcomes, thus leading 
to a reciprocal dependency between agents' activities and the networks they form. 

To give some insights into the interplay between social activities and the under- 
lying evolving social networks, this papers analyzes a model of emergence of coop- 
eration on dynamic networks. We choose the prisoner's dilemma game as a case 
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Table 1: The matrix of the prisoner's dilemma game 
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study since it is known to be the worst situation for cooperative dyadic outcomes, 
compared to other related games such as the Chicken game. 

We determine the conditions for emergence of cooperation from an all defecting 
population and show how heterogeneous population states, with both cooperators 
and defectors, can be stable thanks to emergent structures of the activity network. 
This work builds on a model previously introduced [lj and proposes an alternative to 
the cliquishness or "choice and refusal mechanism" that often stand for explanations 
of cooperation. 

1. The prisoner's dilemma on networks 

A prisoner's dilemma game (PD game) is a two players game which payoffs matrix 
has a structure described by table 1 with the conditions < p < r < 1. 

The prisoner's dilemma game belongs to the class of social dilemmas and has 
become a paradigm for modeling cooperation and altruistic behavior, i.e. situations 
where one achieves an action that benefits to the other at its own expense. The 
dilemma lays in the fact that option D (defect) leads always to the highest reward 
whatever the other does. But when both players choose D } they finally receive less 
than if both had played C (since p > r). This means that mutual cooperation is 
always more profitable than mutual defection (collective rationality), but given the 
opponent's action, defection is always individually more profitable than cooperation 
(individual rationality). The prisoner's dilemma is thus a prototype of situations 
where there is a dilemma between individual and collective rationalities. 

The game can be played several times in a row or only once ; players can play 
simultaneously or sequentially. In this paper, we will study repeated sequential PD 
game. The repeated sequential form of the prisoner's dilemma can be useful to 
model social interactions based on the mutual gift, as described for example in [19], 
or trust ([7], [33]). The recent access to huge databases related to sequential social 
interactions (phone calls, mails, posts on blogs, etc.) constitute also a new source of 
applications of PD game modeling ([IS]), with perspectives of empirical validation. 

Experimental studies based on the prisoner's dilemma game generally conclude 
that the level of cooperation observed in human population is not consistent with 
non-cooperative game theory predictions (|0J; [13] for the sequential form). In par- 
ticular, one of the most puzzling phenomena observed in experimental studies is that 
among the various behaviors encountered, there is often a substantial proportion of 
altruist behaviors, i.e. unconditional cooperators |2]. Behaviors observed in experi- 
mental studies can be roughly categorized into three types: selfish players, who most 
of the time defect, conditional cooperators, who cooperate if the other player did, 
and altruist players, which most of the time cooperate whatever the other did. If het- 
erogeneity in agent's types is addressed in some models, few of them explain how it 
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can be maintained, and even fewer explain how it can emerge from an homogeneous 
population of selfish players. 

Recent literature demonstrated that the topology of social networks can help 
to sustain cooperation (|3] ; [3] ; [H]; [ID]). One of the main conclusions of these 
studies is that cooperation is favored by a high degree of cliquishness of the activity 
networks and the possibility of refusing interaction. This is a motivation to consider 
models with partner selection where the topology and weights of the activity network 
are emergent properties. 

In the following, we will propose a mechanism of endogenous network forma- 
tion that could stand for an alternative to the cliquishness or "choice and refusal" 
mechanism in the explanation of emergence and sustainability of cooperation with 
heterogeneous population. 

1.1. The model: why address books are important 

Let P be a population of N agents playing PD game in discrete-time. The game 
is played sequentially: an agent (the first mover) announce its play (C or D), and 
its partner (the second mover) reacts accordingly. For example, someone can post 
a message on the blog of a friends, and this latter can answer (or not) with a post 
on the first mover's blog. We will assume that at each period, each agent has the 
opportunity to play only once as a first mover (this can be interpreted as a time 
constraint). 

Agents keep in their address book the names of their partners for all previous 
successful interactions as first mover ((C, C) or (D,C) outcomes). Names in agent 
i address book will be called its relationships and agents having i in their address 
book will be called its friends. 

We will consider a complete graph for the network of accessibility (first movers 
can choose any agent in the population, like for mail, comments post or phone calls). 
The network of activity defined by the set of all address books will be an emergent 
evolving directed weighted network. 

Agents can choose their partners to some extent: 

• With a probability (1 — e), e G [0, 1], first movers can choose their partners in 
their address book, thus taking advantage of what they have learned so far. If 
their address book if empty, they are matched at random in the population. 

• With a probability e, first movers are matched at random in the population. 

Parameter e can be interpreted as an uncertainty level in the partner selection pro- 
cess or as an exogenous structural factor (e.g. some competencies an agent needs are 
sometime not present in its address book). We will notice that e = 1 corresponds to 
the classical approach of evolutionary game theory for which is as been proved that 
cooperation is not sustainable. 

Agents start their life with empty address books and do friends and relationships 
during social interactions. 

1.2. The strategies 

A period of the game unfolds as follows: 
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Figure 1: Interactions as first and second movers. Each round, an agent can play several times as 
a second mover, but only once as first mover. 

1. Each agent is matched as first mover with a partner according to rules given 
above, 

2. Each agent does a proposition (C or D) to its partner, 

3. Each agent answers (D or C) as a second mover to every agent that has chosen 
it. 

At a given period, an agent can thus play several times as a second mover, but only 
once as first mover. 

The set S of strategies is constituted of the three types mentioned previously: 
altruist, reciprocator and selfish agents in proportion 7, 5 and 1 — 7 — 5. These types 
are defined by their behaviors as first mover (table 2) and as second mover (table 
3). Their proportions evolve in the simplex A = {(7, + 7 < 1;7 > I; 5 > 1} 
. We will assume that agents know the proportions of the different types in the 
population^ so that they can compute their expected payoffs E as first mover and 
can play accordingly : E(C) = (7 + S)r ; E(D) = (1 — *y)p + (7). The condition 
under which E{C) > E(D) is thus S > » + (1 ~ p ~ r)7 

Although they are stylized types, these strategies will give some insight into the 
phenomena that could potentially stabilize cooperation. 

1.3. The evolutionary dynamics 

At the end of a period, fitness of agents are defined as the sum of all their dyadic 
interactions payoffs at this period. These payoffs are assumed to be linked to a 
reproductive success: agents have a probability (1 — 9) to die at the end of period. 

Each died agent is replaced by a new agent according to a replicator dynamics 
so that the population size is kept constant: with a probability proportional to its 
fitness, one agent is chosen at random in the population, and a new agent of the 
same type with an empty address book is created {overlapping generation game). 



1 More sophisticated model would assume that agents learn these proportions from the outcome 
of their previous interactions 
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Table 2: Behaviors of first movers in function of the state (7, S) of the population. 





First Mover 




Type 


In address Book 
(with relationships) 


random 


Selfish 


Search for an altruist and play D. 


C if 6 > p + ^-p-^t 

r r 




If none play C 




Reciprocator 


Pick a name at random 


C if 5 > p + Ci-p-t-Jt 




and play C 




Altruist 


Pick a name at random 
and play C 


C 



Table 3: Behaviors of agents as second movers. 



Type 


Second Mover 


Selfish 


D 


Reciprocator 


Do what first mover did. 


Altruist 


C 



2. Analytical and computational study 

We will now give some analytical and computational insights into this model. 
One of the stylized fact we want to reconstruct is the existence of areas in the (e,p) 
parameter space where we find stable heterogeneous population with significant level 
of cooperation. This means that the dynamics under study shall have an internal 
attractor in A. 

Given the complexity of the model, an analytical study will necessarily adopt 
some approximations ( e.g. infinite population size assumption). On the other hand, 
computational studies can never guaranty that the number of time-steps considered 
in simulations is sufficiently high to distinguish a very slow evolution toward an 
homogenous population from the existence of an internal attractor. When the vector 
fields tends to zeros and when there no way to assess convergence, simulations are 
poorly informative about internal attractors. 

Our methodology is thus to find an analytical approximation of the model (2.1) 
and check that it fits well the simulated model (2.2). Then understand cooperation 
in the analytical model (3.1), characterize internal attractors (3.2) and study one of 
these internal attractors to understand how network patterns can sustain coopera- 
tion in heterogeneous population (3.3). Then we will see to what extent the choice 
of the strategy set is important to observe these phenomena (4). 

2.1. The fitness function 

We can see from table 2 that behaviors of agents vary in function of the sign of 
5 — -. The part of A concerned by the problem of emergence of cooperation contains 
the point (0, 0) corresponding to an homogeneous populations of selfish agents. The 
interesting area for our study is thus defined by 5 < & + <1 ~ P r ~ r - >7 . For sake of clarity, 
we give in this paper the analytical approximation of expected payoffs for this area 
only. 

Theorem 1. For an infinite population defined by (7,5) G A, with the approxi- 
mation that the proportions of types in population are constant at the agents' time 
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scales, the mean expected payoffs 7r a , 7r r and tt s (for altruists, reciprocators and 
selfish agents respectively) in the domain 5 < 2 + < ~ 1 ~ p ~ r ' >7 are given by: 

w 2p(l -7 + 0(7-1)) +7 (1-0)7 x , , s 

^ " ^ 1 + 0(7-1) + 1 -(!-(/+ W + MP{1 ' 7) + 7) 

ff = (1 _ e /7(r + 2p-l)+ 7 + 2p(l- 7 -g) rj ) +e ( 2p+ 7(l-2 P +r)) 

r 1 6)[ 1 + 0(7-1) + l-(l-(5 + 7 ))0 j+ 1 P+1[ P+ )] 

rdS r(2 7 + 5) , n . 

= (1 - 6)( 1 + 0(7-1) + l-d-^+V + er( ' + ^ 



PROOF. [I] Proof given in append 



IX. 



From this payoffs expressions, the relative fitness for each type is given by: 

/, = (i) 

where < n >= 77r a + 5n r + (1 — 5 — 7)7r s 

Whether cooperation will develop in the population will depend on e, r and p. 
For sake of convenience and clarity we will adopt a standard parametrization of 
the PD game matrix: r = (1 — p). This enables to reduce the parameter space 
to two free parameters e and p which helps for visualization. Intuitively, p fixes 
the strength of the social dilemma: the higher p, the stronger the dilemma. In 
the following, attractors and analytical trajectories are computed by solving these 
equations numerically. 

2.2. Comparison between analytical and computational results 

We first can compare the dynamics of the analytical model and the simulated 
model. We present here the comparison for the parameters: = 0.99, e = 0.3 and 
p = 0.3 (thereafter referred as settings 1). We ran 10 simulations and collected data 
on types' proportions in population, address books and payoffs. Initial conditions 
are (7, 5) = (0.1, 0.1) and iV = 500. Initial rate of cooperation for first move play is 
thus 10%. 

As expected from the analytical study (fig. [2]- circles on the graph), the popula- 
tion converges toward a full cooperative state with a mixed population of altruists 
and reciprocators. Cooperation has become sustainable and selfish agents have dis- 
appeared. 

We can also check that the stationary environment approximation made in propo- 
sition 1 is quite reasonable: the time between two markers (circles) represents one 
generation. The proximity of two successive markers indicates that the environment 
of an agent does not change much during its lifespan. This approximation will al- 
ways be satisfied around an internal attractor since it is the place where the vector 
field tends to zero (the fitness function is continuous in A°). 
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Figure 2: Comparison between analytical predictions and simulations. For e = 0.3, p — 0.3, 
9 = 0.99 the theoretical analysis predicts very well the behavior of the system. Line with circles: 
Analytical prediction of the trajectory in case of infinite population size. Dotted lines: plots of 10 
independent simulations with N = 500. Each circle is separated by 100 periods i.e. the agents' 
mean lifetime expectancy. 

3. Cooperation, social activity and network formation 

We will now see what we can learn from this model and its infinite population 
size approximation about the role of network formation in emergence of cooperation. 

3.1. Cooperation and social activity 

To better understand the reasons of the convergence towards a fully cooperative 
state in settings 1, we can turn toward the analysis of the structure of social inter- 
actions for each type of agent in the infinite population approximation. From the 
analytical study, we can compute the theoretical level of activity for each type in 
function of 5 and 7, i.e. the mean number of interactions per period. This provides 
information about the degree of involvement in social interactions for each type of 
agent. Let's define the relative activity for altruists RA a (resp. reciprocators RA r ) 
as the ratio between the mean number of interactions per period for altruists (resp. 
reciprocators) and the mean number of interactions per period for selfish agents as 
a function of (7, 5). 

Details about the computation of these relative activities are given in appendix. 
We can observe on figure 3 that: a) these two ratios are always greater than one, 
which means that selfish agents are the less active in the social network (they don't 
have friends actually); b) altruists are the most active of all types, especially when 
in small proportion. 

This example illustrates the way altruists can be successful even in presence of 
selfish agents: they are more active than everybody else. Since altruists are the only 
ones to cooperate as second mover, they are the only one to become friends with 
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Figure 3: Theoretical relative activities for reciprocators and altruists (0.1 < 7 < 0.9 and 0.1 < 
S < 0.4 ; e = 0.3 and p = 0.3, 9 = 0.99). Upper surface : relative activity of altruists ; lower 
surface: relative activity for reciprocators 

. The plotted trajectory is the projection on this surface of the theoretical 

trajectory of figure [3] 



reciprocators, and consequently they do relationships faster than other types. Al- 
truists are more popular. The fact that their selfish partners cheat them is balanced 
by the fact that they are more appreciated as second mover partners in the social 
network. 

3.2. Looking for heterogeneous equilibriums 

An interesting question concerning the infinite population size approximation 
concerns the existence of internal attractors. What about the stylized fact presented 
in the introduction: stable heterogeneous populations of altruists, reciprocators and 
selfish agents? Given (e,p,9) e]0, l[x]0, 0.5[x]0, 1[, the replicators dynamics is de- 
termined by the folowing vector field V in the simplex A: 



y = <^V 5 

7717 - 8ir r - (1-5 - 7)7r e 



e.,p 



T/ ' a 

V ^ = — xZ n x ZXZ ~ 7 U P 



77r a - 5n r - (1 - 5 - 7)717 
A point (5,7) G A° fl (]0, l[x]0, ttz^jD 1S an internal attractors only if: 



+ V S 2 = | e , p (2) 

We solved numerically equation [2] to get the set of internal attractors for a given 
value of 9. Figure 4 displays the set of pairs (e,p) for which the dynamics has an 
internal attractor. This set has a non-empty interior and for 9 = 0.99 it is crab's 
claw shaped and oriented along the line p = 0.5 — 0.35e. As shown on the figure, 
the rate of cooperation at the attractor decreases in the population along this line, 
which means that, as expected, e has a negative impact on cooperation. Figure 5 
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Figure 4: Existence of internal attractors in the (e,p) space. The level of gray is indexed on the 
rate of cooperation at the attractor (the darker, the less cooperative). This has been obtained from 
Equ. [2]solved by numerical analysis for 9 — 0.99. The arrow points to the pairs (e,p) corresponding 
to the simulation presented in fig. 6. 

displays the set of pairs (7, 5) that are internal attractors for some value of e and 
p. We can see that most of these attractors are situated above the line y = x, 
which means that there are generally more reciprocators than altruists at internal 
attractors. This is in line with experimental studies. 

Last, we can study the influence of the life expectancy, which is also the ratio 
between the network updating timescale and the population renewal timescale. If 
cooperation disappears for very low life expectancy (e.g. 9 = 0.5), we observe 
cooperation and internal attractors as soon as 6 = 0.9, which means that cooperation 
is quite robust against variations of life expectancy. 

3.3. Patterns in the emergent network 

With this map of attractors from the analytical study, we can look at the struc- 
ture of internal attractors in the simulated model and check, passing, that the ana- 
lytical model predicts well the existence and location of internal attractors for the 
simulated model. For example, a simulation ran for 25.000 time steps for 9 = 0.99, 
e = 0.7 and p = 0.25 (thereafter referred as settings 2 - the arrow on fig. 5) has a 
trajectory that cycles around the analytically predicted attractor, which corroborate 
the existence of an internal attractor despite the hight value of e. Its location is close 
to the one predicted by this analytical insight (fig. 6). 

The analysis of the structure of the emergent social network at this internal 
attractor helps us to understand the interplay between the dynamics of the activity 
network and emergence of cooperation. In directed networks, important vertices can 
be defined in terms of hubs and authorities^] [15]. 



2 If a vertex points to many vertices with large authority weight, then it should receive a large 
hub weight. If many vertices with large hub weight point to a vertex, this latter should receive a 
large authority weight. 
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Figure 5: Location of attractors in the (j,S) space obtained from numerical solution of Equ. [2 
Each point represents an attractor for some value of (e,p) (fig 4.). 




Figure 6: A case of internal attractor (e = 0.7, p — 0.25,8 — 0.99) : For some values of e and p, 
a society starting from an almost all-defecting state (19% of D-first-mover reciprocators, 80% of 
selfish agents, 1% of altruists ) evolves toward an internal attractor with a mixed population of 
the three types. The simulation presented here is 25 000 periods long, corresponding to 250 mean 
lifetime expectancy. The black line represents the theoretical evolution in an infinite population and 
the theoretical attractor is situated in the center of the black square. With finite population size 
(here 1000 agents), the population cycles around the attractor. Asymmetries in the vector field's 
strength around the theoretical attractor as well as variance effects might explain the discrepancy 
between simulation results and predictions. 
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Figure 7: An example of network structure in the case of an internal attractor (100 agents, 9 = 0.99, 
e = 0.7 and p = 0.25). Arrows indicate who is the friend of whom. The graph shows the ten hubs 
and ten authorities of the network (a Pajek plot). Authorities are all of altruist type (A) while 
hubs are all of selfish (E) or reciprocator (R) type. 

In our model, authorities are agents with lots of friends, and hubs are agents with 
lots of hub relationships. The plot of the directed network formed by the address 
books around the attractor (settings 2) reveals an interesting structure: the first 
hubs are always of altruist type, the first authorities are always of reciprocator or 
egoist type. For example, figure 7 represents the ten hubs and ten authorities of a 
network of 100 vertices with 9 = 0.99, e = 0.7 and p = 0.25. All the ten authorities 
are altruist while the ten hubs are all of selfish or reciprocator type. 

This kind of organization of the social network suggests an explanation of coop- 
eration than differs from the cliquishness of cooperators: popularity of cooperators. 
The details of the activity equations in the infinite population approximation sup- 
port this view. We can see that while altruists are more likely to be authorities, 
selfish agents and reciprocators are more likely to be hubs. This emphasizes the 
importance of having the three types of agents to get heterogeneous equilibriums at 
behavioral level. Types maintain for different reasons: altruists because they have 
a lot of friends and among them both altruists and reciprocators; reciprocators be- 
cause they avoid being exploited by selfish agents and still have some friends among 
altruists, selfish agents because they exploit altruists. 

We will notice passing, that this simulation is also a case of emergence of cooper- 
ation from an initial population that had only 1% of altruist agents. In most models 
explaining cooperation, especially models based on cliquishness, sustainability only 
is explained because the model requires that the proportion of altruists passes a 
certain threshold to be sustainable. In these cases, emergence cannot be explained. 
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4. Scenarios for emergence of cooperation 

Our initial aim was to explain the sustainability of altruists in presence of selfish 
agents. We can wonder whether the presence of reciprocators is contingent or nec- 
essary. It might be hard to study this question from a computational point of view 
because, as a matter of emergence, a simulation can only prove that emergence is 
possible (by the presentation of a simulation where emergence took place) but not 
that it is impossible. We will thus turn again toward the analytical model to get 
some insight. 

Theorem 2. For an infinite population of selfish and altruist agents, an all-selfish 
population is an evolutionary stable state (ESS). 

PROOF. Mlf we develop ^altruists — ^seifish\s=o in power 0/7 at the origin we have: 



277(1 Q\ Q 

^altruists ~ ^selfish = -(1 - e) 2ep + (1 - ^)^— -7 + 0(7) (3) 

The first coefficient of this development is always negative and the second always 
positive. This implies that a population of selfish agents is an evolutionary stable 
state (ESS) when selfish and altruist are the only available strategies. 



The consequence of theorem [2] is that emergence of cooperation is never possible 
through introduction of few altruists in large population of selfish agents (except in 
the trivial case p — 0). Nevertheless it often can maintain for some values of e and 
p once this proportion passes a certain threshold. 

Theorem 3. For an infinite population of selfish agents and reciprocators, the evo- 
lution is neutral between the two types in the area 5 < ^ + . 

PROOF. The proof of this theorem is straight forward since for 5 < & + ( 1 ~ p ~ r )^ 
reciprocators and selfish agents have the same rule of behavior and thus 7i r = n s . 



The study of the fitness expression in the domain S > shows that above 
this threshold, reciprocators are sustainable even if they are first mover cooperators. 
Since this threshold is quite high, and given that neutral evolution is a very slow 
process, it is unlikely that cooperation emerges that way in large populations. 

The consequences of theorems [2] and [3] is that the most probable scenario for 
emergence of cooperation starting from an all-selfish population is two folds: 1) a 
slow neutral evolution from a mixed population of selfish agents and few recipro- 
cators ; 2) when there are enough reciprocators, altruists become viable and can 
invade the population. For large populations, these three types are consequently a 
minimal set for emergence of cooperation starting from an all-selfish population. An 
example of such emergence is given by simulation of figure 6. 
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5. Limits of the model 

The modeling activity necessarily simplifies real phenomena. Modelers have to 
make choices, and the resulting models are not necessarily commensurable. However, 
we will highlight some of the limits of our model that future works could eventually 
address, and contrast, in the next section, our results to the existing literature. 

First of all, our model assumes that population size remains constant through- 
out the co-evolutionary process. This is clearly a simplification, and the role of a 
changing population size remains an open problem. 

The model also assumes that agents know the proportions of other agents types 
in the population and take their decision accordingly. Although this is a standard 
assumption, it would have been more realistic to assume that agents learn these 
proportions during their interactions, updating their belief with a bayesian rule for 
example. It is not clear at first glance whether these refinements would be favorable 
or detrimental to cooperation. 

An other issue is the choice of the initial set of strategies. This choice is itself 
influenced by the mode of strategy update (e.g. replicator dynamics vs. imitation 
of the most successful agent). We have considered here a set based on the two most 
common rules, All-D and All-C, and we can question the stability of our results 
toward a modifications of the strategy space and interactions procedures. There have 
been a huge variety of strategies proposed so far in PD games and there is few chances 
that we could get general results about stability of cooperation independently of a 
precise definition of the strategy space. For example, [12] demonstrated t that the 
outcome of a spatial prisoner's dilemma was undecidable: we know that there is no 
general procedure to decide whether a given strategy will invade a given ecology. The 
most important conclusion of theorems 1-3 is thus: in presence of altruist and selfish 
agents, a minimal set for emergence of cooperation has at least three strategies. 

Nevertheless we can provide some intuitions about the stability and robustness 
our result concerning emergence of cooperation against some alternative choices. 
First, we can wonder whether cooperation would still be sustainable in case of a 
simultaneous PD. Simultaneous PD would require completely different settings since, 
for example, the implementation of a reciprocating strategy would imply that agents 
have at least a one step memory for all interaction outcomes (in our model agents 
only keep in memory successful first move interactions and use the sequential aspect 
of the interaction to eventually reciprocate when playing as second mover). But 
as highlighted before, the success of cooperation is due to the high popularity of 
altruists. This means that the important thing is that altruists can be recognized by 
other altruists and reciprocators in first move interactions so that offspring of these 
two populations make friends quicker than other types. This is still compatible with 
simultaneous PD, since it is possible to detect altruistic behaviors in simultaneous 
PD game. Nevertheless, further studies are needed to confirm this intuition. 

Second, we can wonder whether cooperation would still be sustainable if we were 
to introduce errors in implementation of actions. It is well known that this kind of 
errors is detrimental to cooperation. However, we conjecture that while errors will 
certainly decrease the rate of cooperation in a population, cooperation will still be 
sustainable for a large domains of the parameters space when the level of errors is 
no too high. In our model, agents are bound to meet several cooperators because 
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of errors in the partner selection process. Consequently their address book quickly 
has few names. Errors on implementation of actions will certainly shorten this 
list of relationships, by discarding cooperators that would have played D by error. 
However, if not too frequent, they should not qualitatively change the distribution 
of non-empty address books and preserve patterns of interaction. However, the 
interplay between errors on actions, errors level on partners selection and cooperation 
is highly none trivial and is worth being further investigated. 

Last, we can question the choice of a replicators dynamics for the evolution 
of agent's types. The replicator dynamics is a standard in the modeling of social 
dilemma. Although we can question its relevance in the modeling of cooperation in 
the case of human species (see for example [5]), it is a good framework to study the 
role of emerging social structures in situations of social dilemma where cooperation 
is a dominated strategy. 

6. Comparison with related work 

We have presented here a model of emergence of cooperation in evolutionary 
games that highlights the role of network formation and effects of network structure. 
In line with empirical data, it proposes a mechanism that explains the persistence of 
heterogeneous types (heterogeneity in rule for changing behavior), and in particular 
the sustainability of altruism (presence of unconditional cooperators) even in case 
of strong social dilemma. We used a hybrid approach with both analytical and 
computational insights. We will now discuss the contributions of our model in the 
light of these different points. 

Emergence of cooperation occurs when a population of cooperators invades a 
population of defectors. This is an important issue, both from the methodological 
point of view - models of cooperations should not assume a priori that cooperation 
is already there -, and from an empirical point of view - lots of natural ecosystems 
are thought to have evolved from non cooperative states (unicellular organisms to 
multicellular, individuals to societies, etc.). Many explanations of cooperation rely 
on a cliquishness effect, which holds only once the proportion of cooperators is suf- 
ficiently large. For example, [T7], [16], [10], [10], [UJ and [23] consider evolutions 
where the initial proportion of cooperators in the population is at least 50%. Thus, 
these models, without further results, are not appropriate to study the origin of 
cooperation in natural systems. On the contrary, our model proposes a clear sce- 
nario for emergence of cooperation, which moreover predicts different steps for the 
resilience of cooperative states: a slow neutral evolution with the introduction of 
reciprocators followed, by a faster evolution with the introduction of altruists. 

Another nice property of our model is that it explains cooperation even in cases 
of strong social dilemma where the temptation to defect is high (p > 0), i.e. for 
parametrization of the social dilemma game where both the ratio between (D,C) 
vs. (C,C) payoffs and (D,D) vs. (C,D) are high. For example, in [16] or [22], the 
dilemma is weak since the difference between (D,D) v.s (C,D) payoffs is an e 0. 
In other models (e.g. the seminal model of [211]), cooperation collapses as soon the 
ratio between (D,C) payoffs and (C,C) payoffs is too high. 

Network formation and effect of network structure is a quite recent concern in 
social dilemma modeling. We can distinguish models that consider static networks 
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with different topologies ([T2], [TB], [22], [II], [IE], 0, |25]) from models that take 
into account networks dynamics. 

Among these later, one of the most common process considered for network for- 
mation is the choice and refusal mechanism ([3], [8], pf], [B], [HI], [23]). Players 
select their partners in function of the productivity of the interaction and can refuse 
to interact with a player if the expected associated payoffs fall below a given thresh- 
old. In those models, the main conclusions is that selection of interactions with 
choice and refusal enables cooperators to interact preferentially with other coop- 
erators, which provides the network with the appropriate level of cliquishness to 
sustain cooperation. This contrasts with our explanation in terms of popularity of 
cooperating agents. 

This explanation is related to the number of interactions per agents rather than 
to the possibility to form cliquish communities. A similar explanation have been 
proposed in [21]. However, there are substantial differences with our approach. 
[2T] consider a mixed population of agents of types A and B that can create or 
prune links at different rates, and interact with all the agents they are linked to 
at a given period. At each time step, a link between an agent of type i and an 
agent of type j is created (resp. pruned) with probably (resp. fiij). Thus, this 
approach can be said link centered whereas our can be said agent centered. The 
advantage of the agent centered view is that it takes into account time constraints. 
Think for example at sending personalized e-mail. It takes time to interact and 
one cannot take an arbitrary number of initiatives within a limited period of time. 
This is the reason why we choose to limit to one the number of interactions for 
first movers. The second important difference with [2T] is that the authors propose 
results concerning sustainability of cooperation in the domain where the timescales 
between the network update and the strategy update are very different (9 <C 1 or 
9 « 1). This enables to neglect the process of link formation through time. Thus, 
they conclude that "the linking dynamics introduces a simple transformation of the 
payoff matrix" of the prisoner's dilemma game. We saw that, at least in our case, 
this approximation does not hold if the condition (9 <C 1 or 9 ps 1) is not met. In 
that case, it is necessary to take into account the ontogeny of agents' networks, and 
expected payoffs are complicated functions of 5, 7 and 9. 

Last, contrary to all the above models, our approach introduces errors or per- 
turbations in the choice of partners through the parameter e. Up to our knowledge, 
this dimension of the interactions between agents has not been addressed in previ- 
ous models. For example, this could bring new insights into mobility effect (high e 
being correlated with high mobility): contrary to [23], we would conclude that high 
mobility is detrimental to cooperation. 

To conclude this section, we would like to finish on a methodological point. Most 
approach mentioned so far present either computational studies or analytical solu- 
tions of a simplified version of the initial problem. For this study, we pay a particular 
attention to highlight the convergence between simulations and analytical insights. 
Such a dual approach is desirable in case of models that might present internal ESS. 
Indeed, in case of heterogeneous populations under slow evolutionary process (which 
is typically what happens around an internal ESS), computational models alone are 
poorly informative as long as there is no way, after a finite number of iterations, to 
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assess how close a simulation is from the attractor. With a computational approach 
alone, one cannot be sure that the heterogeneity observed in the population is not 
an artefact due to an insufficient number of iterations. In these cases, analytical 
insights give supplementary evidences against the artefact hypothesis and strongly 
supports the understanding of the model. 



7. Conclusions 

We took the well-known paradox of the PD to illustrate the importance of net- 
work formation in the understanding of social phenomena. We proposed a models 
where heterogeneous agents modify their neighborhoods according to their types, 
while the global network configuration determines the fitness of the different types. 
This co-evolution between agents' types and network patterns leads to characteristic 
network structures that depart from the traditional cliquish structures that account 
for heterogeneous cooperative equilibria. In our case, it is an increase in activity 
of popular (cooperative) agents that explains the emergence and sustainability of 
cooperation. 
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A. Mathematical Appendix 

A . 1 . Proof of theorem [7] 
Proof. Q] 

We will note A = {(7, S)\8 + 7 < 1; 7 > 1; S > 1} and the number of agents 
of type i. To compute the mean payoffs of each type we need first to compute the 
probability for an agent to be connected. 



Lemma 4 (Theorem [T]). Let Pi be a sub-population of agents that do relationships 
(out-coming links) only with agents from a sub-population P 2 in proportion A. The 
probability that a random agent i G P has already a relationship in P 2 is P CO nnectW = 

1 i-(l-A)0 

PROOF. Let i be an agent of age t in Pi. The probability it hasn't met any agent 
from Pi during its life is (1 — A)' . Consequently, the probability that agent i of age 
t hasn't any relationship in P2 is 1 — (1 — A)'. (We neglect here cases where the sole 
relationship of an agent dies before it meet an other agent from Pi. This is a good 
approximation as soon as e.O.X is not too small). 

The distribution of ages in the population has the generatrice function f(z) = 
(1 — 6) J2t>o^- zt - Then the probability for an agent to be connected has the gen- 
eratrice function $(z)(l — 1 _ ( . 1 _ 1 ~^ 6da ^ z + 1 — wjz^l^aw)' ^ e deduce the mean 
connectedness with agents of the considered types(s): 

-y Q 

Pconnect(^) — 1 ~~ ^ _ ^ _ 
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and its variance is : 



Var connect {\) = (1 - — )- 



\)6 J l-(l-\)6 

In the area 5 > ^ + < ~ 1 ~ P r ~ r ' >7 , the domain concerned by emergence of cooperation, 
only altruists cooperate first move. Consequently, they are the only one to be able 
to get new relationships with altruist and reciprocators. Their probability to have 
a non empty address book is then P CO nnect(l + 5). Selfish agents and reciprocators, 
on the contrary, can only do relationships with altruists. Their probability to be 
connected is then P CO nnect{.l) ■ We can now compute the mean payoffs for each type. 
Payoffs terms are composed of two independent terms: the expected payoffs as first 
mover and the expected payoffs as second mover. For each of them we have to treat 
separately cases where the agent is connected, cases where it is not, and exploration 
moves. We thus get: 

n selfish = P cormect (7)[(l - e) + e.p(l - 7) + e.7]... #1 

+ (1 - Pconnectt{l))[p.{l - #2 
-^connect 

+ (1 - Pconnect(l)) ■(! — l) -V + (1 Pconnect 

With : 

• #1 : i is connected, lrst move interactions 

• #2 : i is not connected, lrst move interactions at random 

• #3 : 2nd move interactions with connected agents 

• #4 : 2nd move interactions with not connected agents 
After simplifications we get: 

-.-<>- ^ 1 -; , :^ >+ -' + r^fe> + *<* - * + * 

Similarly we have : 

Kreciprocator = Pconnect (l) ( (1 ~ e) X + C (7 + p(l - 7) ) ) + . . . #1 

+ (1 Pconnect 

( 7 ))(p.(i- 7 ) + 7 ) + ... #2 

+ e. {Pconnect (l) .(1 - l) -P + P C onneet{l + 5).[(1 - e).r. 7 /( 7 + 5) + e. 7 .r]) + ... #3 

+ (1 Pconnect ( 7 )).(1 - 7)-P + (1 - ^co„„e Ct (7 + «))-7-r #4 



After simplification we get : 

w#7(7 + 2 P - 1) + 7 + 2p(l - 7 - 6) r-f , „ 

= ^ - ^ l + g(7-l) + l-(l-(^ + 7))^ + ^ + 7(1 - ^ + 0) 

And finally : 

^altruists = Pconnectil + d)[{l - e).r + e.T.^f + d)] + ... #1 

{I- Pconnect{l + d)).{d+ 1 ).r+ ... #2 

Pconnecth) ■(! - e).r.d/g + Pcconnect(~f + d) .[(1 - e).Tvy/(7 + d) + e.j.r] + ... #3 

(l-i > connect(7 + d))-7-r + ... #4 

After simplification we get : 

= (i - e)( i +g(7 -D + i-d-VV + + 2 ^ 
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A. 2. Theoretical activity 

To compute the theoretical mean activity, we follow the same guidelines as for the 
expected payoffs, the difference being that we have to take into account interactions 
with null payoffs. 

The mean activity of agents can be split into two independents terms: the mean 
activity as first mover and the mean activity as second mover. For each of these 
terms, we have to treat separately cases where the agent is connected, cases where 



it is not, and exploration moves. We thus get: 

Act s = 1 + ... #1 + #2 

e.[P connect^). (1 — 7) + Pconnect(~f + 6).j] + ... #3 

(1 - P 'connect^)). (1 - 7) + (1 - Pconnect(-y + S)).j + ... #4 



• #1 : i is connected, lrst move interactions 

• #2 : i is not connected, lrst move interactions at random 

• #3 : 2nd move interactions with connected agents 

• #4 : 2nd move interactions with not connected agents 
Similarly for reciprocators: 



Act r = 1 + ... #1 + #2 

+e 

•■^connect 

( 7 ).(1 - 7) + P connect (l + *)•[(! - e)-7/(7 + 8) + e/y]) + - #3 

+ (1 - P, connect (l) )•(! - 7) + (1 ^connect 

and for altruists: 

Ad a = 1 + - #1 + 

+P C onnect{l)\e.{l - l) + (1 - e).d/~f] + P C onnect{l + 5). [e/y + (1 - e). 7 /( 7 + 5)] + ... #3 
+ (1 - ^connect (7)) •(! _ 7) + (1 P connect ( 7 + <5)). 7 +... #4 



The relative activity of altruists (resp. reciprocators) is thus: RA a = (resp. RA r = ^ff 21 )- 
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B. Computational Appendix 

Lets P(t) be the population at time t. We give here the algorithm used for the simulations. 

Parameters and notations The parameters are: 

• e S [0,1]: Exploration parameter 

• (p, r) <G [0, t,] 2 : parameter of the social dilemma 

• € [0, 1]: parameter defining the mean lifetime expectancy. 

Initial conditions : N agents with empty address books from the three types: altruist, 
reciprocator and selfish, in proportion 7 ,5 and (1 — S — 7) . 

At each period of time t : 

Set all payoffs to 
Interaction (parallel updating): 

1. For each agent in P(t), a binomial random variable of mean (1 — e) is drawn to determine if 
its partner will be chosen in its address book or at random in the population. In all cases, 
if its address book is empty, the partner is chosen at random. 

2. In case of interaction as first mover with an unknown partner, the partner is chosen at 
random and the action is determined according to table 2. 

3. In case of interaction as first mover in address book, the action is determined according to 
table 3: 

• Altruists and reciprocators choose a relationship at random and play C. 

• Selfish look for an altruist and defect. If they only know reciprocators, they choose 
one at random and cooperate. In all other cases, their interact with an agent taken 
at random in whole population 

4. Each agent plays as second mover according to its type (c/. table 3). 

5. Each first mover adds the name of its second mover partner in its address book if the partner 
has cooperated. Selfish have special categories: altruist (obtained after a (D, C) results) - 
reciprocator or altruist (obtained after a (C, C) result) - reciprocator (obtained after a 
(D, D) result after interaction with an agent of the category reciprocator or altruist. 

Selection : 

1. After interactions took place, the fitness of each agent is given by the sum of all dyadic 
interactions payoffs for this period. 

2. Each agent dies with a probability (1 — 8) . 

3. Died agents are replaced with new agents with empty address books. Died agents are 
removed from the address books of their friends. This leads to the population P(t + 1). 

4. Types of new agents are determined by a replicator dynamics indexed on the total payoffs 
of agents from P(t): for each new born agent, an agent is selected in P(t) with a probability 
proportional to its fitness. The new agent inherits the type of this particular agent. 
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